write_durable fsync'd file content but never the containing directory, so
on POSIX the renamed directory entry (and the rename itself) was not durable
across a power cut. Since optimize() collapses to one segment and deletes the
old ones, losing the flip/dirent could leave the index unreadable. Add a
best-effort, cfg-free sync_dir (real dir-fsync on Unix; File::open on a dir
fails harmlessly on Windows, which relies on NTFS journaling), call it after
write_durable's rename and after the segments.gen flip in
write_optimized_generation, and correct write_durable's overstated doc
comment. Incremental commit path left intentionally non-durable.

The near-stopword bounded-memory property (peak heap independent of a
term's doc_freq) was only measured by the manual optimize_bench binary, not
in CI - a regression to materializing whole terms would pass all tests. Add
a tracking-allocator integration test that merges the same multi-segment
input through both merge_segments_streaming and the batch merge_segments and
asserts the streaming peak is >3x below the batch peak (measured ~6x). A
global allocator in a tests/*.rs file only affects that test binary.

merge_streaming_inner wrote the final {merged_name}.cfs directly, so a
mid-write I/O error (disk full, etc.) left a truncated final .cfs on disk;
the cleanup only removed the .fdt/.frq/.prx temp files. Assemble into
{merged_name}.cfs.tmp, fsync, then atomic-rename onto the real .cfs and
fsync the directory, so a failure can only leave the staging file (now also
cleaned on both paths), never a partial final .cfs. Byte-identity tests
(which read _m.cfs) already cover rename correctness; added a regression
test that the staging file and final .cfs are clean when the rename fails.

A strict failing-first test for the mid-write I/O error itself needs a
fault-injection seam (test-only production code), so per 'TDD when possible'
this path is covered by the atomic-rename design + the reachable rename-fail
test rather than an injected mid-write error.

optimize() deleted the superseded segments' .cfs/.sti/.del immediately after
the flip, but the prune deliberately KEEPS the previous generation's manifest
as a grace window for lock-free readers. That manifest still referenced the
just-deleted files, so a reader that opened the previous generation hit
dangling references; on Windows the unlink of a reader-mapped .cfs also failed
silently and leaked. Defer deletion one generation via a best-effort
'pending_deletions' sidecar: each flip reclaims the previous round's superseded
files (whose referencing manifest the prune just removed) and records this
round's for the next flip. A bug here can only leak disk, never delete a
still-referenced file. Grace-window and reclamation covered by tests.

# Conflicts:
#	sdsearch-core/src/zsl/writer/index_writer.rs

…ndex, differential-tested)

…al test

…ed, incl. large corpus)

…ial-tested incl. large corpus)

…the whole .tis; eager kept as a test-only differential oracle)

…e-seek the term dict per term

Re-measured native + sdsearch (all sizes, 20 iters, fresh .so with the
merge-cursor TermInfo fix). Story:
- rebuild back to the pre-lazy baseline (native 500k 7.27s vs 7.22s;
  sdsearch 500k 9.84s vs 9.71s) — the ~40% optimize()/merge regression the
  lazy reader introduced is eliminated.
- search few/none now BEAT Zend at scale (100k few 20.8x, 500k few 8.4x)
  where the eager reader lost 0.3-0.4x; many unchanged; churn improved.

Zend rows are carried forward unchanged (engine untouched): 1k-100k verbatim
from the prior run, 500k reconstructed from the committed RESULTS.md values
(the report renders only ms/rss/p50/hits, all previously measured).